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Abstract 

Ensuring sufficient liquidity is one of the key challenges for designers of prediction mar- 
kets. Various market making algorithms have been proposed in the literature and deployed in 
practice, but there has been little effort to evaluate their benefits and disadvantages in a sys- 
tematic manner. We introduce a novel experimental design for comparing market structures in 
live trading that ensures fair comparison between two different microstructures with the same 
trading population. Participants trade on outcomes related to a two-dimensional random walk 
that they observe on their computer screens. They can simultaneously trade in two markets, 
corresponding to the independent horizontal and vertical random walks. We use this experi- 
mental design to compare the popular inventory-based logarithmic market scoring rule (LMSR) 
market maker and a new information based Bayesian market maker (BMM). Our experiments 
reveal that BMM can offer significant benefits in terms of price stability and expected loss when 
controlling for liquidity; the caveat is that, unlike LMSR, BMM does not guarantee bounded 
loss. Our investigation also elucidates some general properties of market makers in prediction 
markets. In particular, there is an inherent tradeoff between adaptability to market shocks and 
convergence during market equilibrium. 

1 Introduction 

Interest in prediction markets has increased significantly in recent years across academia, policy 
makers, and the private sector [20J, HU [IJ [5] . Wolfers and Zitzewitz [19] discuss how prediction 
markets have gone from minor novelties to serious platforms that can have substantial impact 
on policy and decision-making; prediction markets are now accepted as information aggregators 
that produce quantitative forecasts. Companies like Google, Microsoft, and HP have deployed 
prediction markets internally for forecasting product launch dates and gross sales. Prediction 
markets have often outperformed opinion polling: for example, the Iowa Electronic Markets have 
usually outperformed opinion polling in predicting US political races [2]. There is little doubt that 
prediction markets are valuable for information aggregation for two reasons: (1) They produce 
meaningful quantitative forecasts; (2) Those who possess information are better incentivized and 
held accountable than they are in alternative information-gathering methods like surveys or polls. 

Wolfers and Zitzewitz identify five key challenges to the success of prediction markets pjj] . First 
among these is liquidity provision - can prediction markets attract sufficient uninformed trading 
to be liquid and attractive to those with information? Liquidity is the classic chicken- and-egg 
problem, in which some liquidity begets more liquidity. Historically, financial markets have often 
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used market makers to provide initial liquidity to get the ball rolling; financial exchanges often 
provide specific incentives for firms to become market makers. Prediction markets have adopted 
the same idea. Typically, in prediction markets, the market maker is allowed to take on a loss, 
subsidizing the market, to facilitate more liquidity and faster price discovery; this loss is taken as 
a cost of operation. Robin Hanson suggested a family of inventory based market makers based on 
market scoring rules. Of these, the one based on the logarithmic market scoring rule (the LMSR 
market maker) is now the de facto standard for subsidized prediction markets |12j . If the main 
purpose of the prediction market is not commercial, loss-making market makers can make a lot of 
sense, but as prediction markets become less experimental, the subsidies become a real loss which 
must be minimized. 

The LMSR market maker is appealing on several levels: (1) it has a strong guarantee on the 
amount of loss it can suffer; (2) since it is purely inventory based, it is difficult to manipulate in 
some settings [3J; (3) it can be shown that, under certain conditions, particularly that participants 
are rational and learn from prices, a market mediated by an LMSR market maker will converge 
to the rational expectations equilibrium price [T7J. However, the LMSR also suffers from several 
drawbacks, some serious: (1) Although it is bounded, the market maker does typically run at a loss, 
which can be large, (2) A single parameter, b, controls many different aspects of the market maker's 
behavior, including the loss bound, the level of liquidity in the market, and the rate of adaptivity 
to market shocks; setting b to optimally manage these tradeoffs is considered something of a "black 
art" |16j : (3) when the posterior belief of the trading population does not converge (which is likely 
when people have independent information and valuations), the price does not converge to a well 
defined probability estimate, instead fluctuating about the equilibrium price; the fluctuations are 
asymmetric and again sensitive to the choice of b, making it difficult to extract a quantitative 
probability estimate; (4) The market maker provides only point probabilities over outcomes and 
cannot be easily coupled with a measure of uncertainty; (5) the market maker cannot easily be 
applied to unbounded markets. 

An alternative to inventory based market makers is an information based market maker. The 
seminal paper of Glosten and Milgrom introduced a model of market making under asymmetric 
information. Building upon this model, Das O [7] and Das and Magdon-Ismail [8] have described 
efficient market making algorithms for zero-profit (competitive) and profit maximizing (monopolist) 
market makers. These market makers address some of the drawbacks of the LMSR market maker. 
Specifically, in stylized market models where a single shock to the value occurs, the price converges 
rapidly to an equilibrium price, without expected loss; further, the markets need not have bounded 
payoffs. The drawback of these information based market makers has thus far been that after quick 
convergence following an initial market shock to the true value, the convergence after a subsequent 
market shock is slow, because the market maker gets "overconfident" after initial convergence^] 

The behavior of the LMSR and information based market makers are discussed further in 
Section [2j One usually compares market makers either theoretically (eg. [5]) or by using simulation 
in some stylized model of trading (eg. [7]). All the properties discussed above are evident within 
such stylized models. There has been little systematic exploration of the performance of various 
market makers in real settings, where trader behavior is unpredictable. In this paper we present 
a new information based market maker which is able to adapt to multiple market shocks, and 
evaluate it using a novel experimental design for comparing market microstructures in live trading 
experiments with human traders. 

1 If the subsequent shock occurs at time t, the time to converge to the new equilibrium value is exponential in t. 
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Contributions We introduce a new information based market maker which builds from the zero 
profit market maker in [6l [8] - we call this market maker BMM (for Bayesian Market Maker). The 
main innovation is the ability to adapt to multiple market shocks. BMM provides liquidity by 
adapting its spread based on its level of uncertainty about the true value. This allows it to achieve 
small spreads in equilibrium-like states, while remaining adaptive to shocks by increasing its spread 
when the information content in the population changes. 

To adapt rapidly to multiple shocks, BMM can increase its uncertainty parameter exponentially 
quickly. It does so by constantly comparing the probability of observed trader behavior over the 
recent history under its current uncertainty with that under the increased uncertainty. This ability 
to adapt to shocks has a drawback. There is always a random chance that recent history will cause 
BMM to increase its uncertainty level, and hence increase spreads, even when there is no actual 
shock to the market. This means that convergence to the true value during an actual equilibrium 
will occasionally get interrupted by chance fluctuations; however these chance fluctuations are 
minor compared to the (typical case) fluctuations of the LMSR market maker. Nevertheless, these 
fluctuations represent a real tradeoff between the tightness of convergence during an equilibrium 
and the ability to adapt quickly during a market shock. By tuning the extent of recent history 
used in determining BMM's uncertainty level, the market designer can quantitatively control this 
tradeoff. 

In simulations, as well as in real trading, BMM can provide substantial benefits compared to 
the Hanson LMSR market maker in many situations, with the caveat that it does not provide a 
similar guarantee on maximum loss - thus, there are situations where its performance (with respect 
to loss incurred) can be worse; such situations are extremely adversarial, however. As with the b 
parameter in Hanson's LMSR, the dynamics of the uncertainty level in BMM controls the tradeoff 
between how adaptive the market maker is to changes in market conditions and its potential loss. 
However, the nature of this dependence is different. 

Our second main contribution is to develop an experimental paradigm for comparing market 
microstructures. In particular, we apply this to compare BMM and Hanson's LMSR market maker 
in a live trading setting. Two challenges one faces with live trading are: 1) the same group 
of traders cannot be used first in an experiment with one market maker, and then in a second 
identical experiment with a second market maker. This is because traders get primed, and even 
if the experiments are identical, the results are incomparable; 2) the same experiment cannot be 
run on two separate groups of traders with a different market maker in each group, because the 
high variability in human traders results in a very high variance due to the small size of any such 
experiments with human traders. The ideal experiment simultaneously tests both market makers 
on the same population of traders in a symmetric way. 

We present a novel experimental design which can capture many aspects of the way information 
is continuously revealed to traders, in addition to allowing for market shocks with and without 
visually perceptible cues. Our design is based on a graphical 2-dimensional random walk which 
simulates the classic Gambler's Ruin problem and allows us to symmetrically compare different 
market structures. We demonstrate the design with several experiments. Our experimental results 
confirm our results from simulation, namely that even in a real setting, with unpredictable human 
traders, BMM outperforms Hanson's LMSR. 

In summary, this paper introduces a new market making algorithm, BMM, that has great 
potential for prediction markets. It offers two compelling advantages over the Hanson LMSR 
market maker. First, at any point in time, it provides a meaningful and useful distribution for 
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the probability of an event occurring. Second, in expectation, it makes less loss while providing an 
equally liquid market (however, the corresponding disadvantage is that it is not loss-bounded). The 
development of this algorithm also provides strong evidence that there is a fundamental tradeoff 
between the reactivity of a market maker to changes in market conditions and the amount of 
loss it can be expected to suffer. Despite the complex underlying mathematics, the ultimate 
implementation of BMM is simple, computationally efficient and can be succinctly described. 

We demonstrate the benefits of BMM in several experiments with human participants. In doing 
so, we also introduce a new family of experiments that can be used to compare market makers (or, 
indeed, entire market microstructures) in a fair, symmetric manner. These experiments can be 
extended far beyond the present work. 

Paper Outline We continue next with a brief introduction to inventory and information based 
market making. We then discuss our new information based market maker BMM which can adapt 
to multiple market shocks. We compare the market makers in both simulation and live trading, 
and conclude with some open questions and interesting avenues for future work. 

2 Market Making 

The key challenge in most prediction markets is inducing sufficient liquidity. How can one incen- 
tivize participants with good information to trade? Without uninformed traders to make profit off 
of, informed traders will not trade (the No- Trade theorem of Milgrom and Stokey [14J). A means of 
creating "uninformed" (or less informed) trades and thereby providing liquidity in modern online 
prediction markets is through automated market making algorithms \12\ [T7] . A market maker is 
an intermediary willing to take the other side of every trade, buying (resp. selling) when someone 
wants to sell (resp. buy); the market maker sets the prices, which will affect whether the trade will 
actually execute or not. We consider a pure dealership market, where a market maker takes one 
side of every trade. An apt comparison could be the foreign exchange desks at airports, typically 
a monopoly for Travelex, who get to set bid and ask prices for foreign currency transactions. This 
model of the market allows us to compare market makers in a fair and precise manner, but in the 
future it will be important to consider integrating market makers with limit order books (which 
poses more of a challenge for evaluation than design). 

2.1 Inventory-Based Market Making 

Hanson describes a market maker for combinatorial prediction markets [12] , which we briefly review 
here in the context of a single market. Hanson's technique adapts the idea of a scoring rule to a 
prediction market setting. While many different scoring rules are possible, Pennock reports that in 
practice the logarithmic scoring rule is the most useful The market maker will take the opposite 
side of any order at a price specified by the market maker. This price depends on a parameter 
b and the market maker's current inventory qt, where t indexes the arrival of trade requests; the 
inventory starts at zero, qo = 0, which corresponds to an initial price of 0.5. The market maker 
sets prices so as to guarantee bounded loss, no matter what the true liquidation value is. 

^http : //blog. oddhead. com/2006/10/30/ implement ing-hansons-market-maker/ 
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Specifically, the spot price is given by 



P(lt) 



e 1t/b 



1 + e^/ b ' 



At time t + 1, if a trade arrives for quantity Q, the cost of the trade (to the trader) is given by 



The volume weighted average price is \C(Q; qt)\/q, and it corresponds to the trader accruing a 
position of size Q using infinitesimal increments, paying the prevailing spot price at each increment. 
If the trader accepts the trade at this average price, then the market maker updates its state to 
qt+x = qt + Q- Since the starting inventory qo = 0, it is easy to verify that the maximum loss 
incurred by the market maker is limQ_ >00 Q — C(Q; 0) = b In 2. 

The parameter b is the only free parameter in the LMSR market maker; not only does it bound 
the loss of the market maker, but it also controls how adaptive the market maker is. If b is small, 
the market maker is very adaptive, taking on small loss; b also controls liquidity in the market. An 
adaptive market maker leads to large bid-ask spreads, implying less liquidity. 

It is known that a market mediated by the LMSR market maker can yield a rational expectations 
equilibrium if traders incorporate information from prices into their beliefs in a rational manner. 
However, consider what happens in a case where a large trading population continues to maintain 
somewhat different beliefs, and some traders regularly come in and trade some typical trade size Q. 
The bid-ask spread 5(Q) for quantity Q, given the current inventory q t , is the difference between 
the average price paid for buying Q shares versus selling Q shares; 



At market inception (qt = 0), the spread is decreasing in b, so higher b means more liquidity (in 
general the relationship between liquidity and b is not monotonic). Suppose the equilibrium price 
corresponds to an inventory q eq ; if typical trade sizes are Q, then the spot price fluctuations around 
this equilibrium have magnitude 



These fluctuations are asymmetric about the equilibrium and persist, making it hard to extract 
a quantitative probability estimate. The choice of b is an important open problem; smaller b 
guarantees smaller loss, but a less liquid market with higher fluctuations around the equilibrium. 
Figure [I] compares the LMSR market maker to an information based market maker (which wc 
discuss next) in a highly stylized model. As we see, the LMSR market maker is adaptive but non- 
convergent; the information based market maker ZP [6, 8j is convergent, but only slowly adaptive; 
a slowly adaptive market maker will incur large loss. One of our goals is to improve ZP to have 
better equilibrium convergence (less fluctuation) than the LMSR, while still being able to adapt 
quickly. 
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(a) Equilibrium behavior in a stable market. (b) Adapting to a market shock. 

Figure 1: Behavior of the Hanson LMSR market maker and the Bayesian information based market 
maker ZP [6J, [8] , for an idealized setting. In both experiments, trades are a fixed size, and traders 
receive a value signal on the basis of which they trade. In (a) the trader value signals are distributed 
about a stable mean; in (b), there is a jump in the mean. For the LMSR market maker, the 
parameter b was set so that the average spread of the two market makers matched. In (a), we see 
that both market makers rapidly arrive at the true value, but LMSR fails to converge, whereas 
the information based market maker converges. From (b) we see that the LMSR market maker is 
very adaptive to a market shock whereas the Bayesian information based market maker can only 
adapt very slowly because it becomes overconfident. These figures are based on simulations where 
traders have beliefs drawn from a Gaussian distribution (the mean of which jumps in the second 
case), and trade in fixed size blocks. 



2.2 Information-Based Market Making 

Though the LMSR market maker is loss-bounded, being purely inventory-based, it is an extremely 
uninformed trader; it typically will substantially subsidize the market, taking on large loss. Al- 
ternative market making schemes necessarily incur more risk. In Hanson's words, "a computer 
program with less than human intelligence that attempts to make markets runs the risk of being 
out-smarted by human traders" [13J. This is because a market maker who makes offers to buy 
and sell any security runs the risk of losing out to either better informed or smarter traders. At 
the same time, "smart" market making algorithms may be able to exploit human trader errors or 
over confidence. Thus, it might be possible to provide liquidity without substantial loss. 

The market maker which we present is information based, and builds on the zero profit market 
maker in [6J, [Hj . We first briefly describe this model of market making, postponing the details to 
Section |4| We start from the canonical Glosten-Milgrom model of price-setting under asymmetric 
information At time t, the market maker has some belief (prior probability density) about the 
value of the security pt(v) (which is the mean of the distribution on trader valuations). We assume 
Po(v) is correct, so the realized value V ~ Pt{v)- An arriving trader gets a signal s drawn from 
a distribution whose expectation is V; the variance of s measures the uncertainty in the trader's 
signal (or information set). The market maker's only information is its prior belief on V. Hence the 
information available to market maker and trader are different, and this information asymmetry 
can be measured by the information disadvantage of the market maker, the ratio of the variance in 
the market maker's prior belief and the trader's uncertainty. This information disadvantage plays 
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an important role in the market maker's actions. 

Given this initial setting, the market maker must set a bid and ask price, and the trader will 
trade accordingly: if s < bid, the trader will sell, and if s > ask, the trader will buy. In a 
competitive setting, the market maker will set prices so as to receive zero expected profit. To do 
this, the market maker solves two non-linear fixed point equations, 

ask = E pt r v \[v\s > ask], 
bid = E n ( v \[v\s < bid]. 

The sequence of papers [6j 18] extend this model to the sequential setting with a Bayesian learning 
market maker. After setting prices, the market maker can now observe what the trader does (buy, 
sell or no trade). This gives the market maker information regarding the trader's signal s, and 
hence information regarding the realization V. Thus, the market maker can update its prior beliefs 
Pt(v) to pt+i(v) to incorporate this new information. The market maker is now ready for the next 
trader. 

The learning market maker in the sequential model is composed of two related parts. The first 
maintains the belief distribution on the value of the market, pt(v); the second sets prices to achieve 
some goal, for example zero expected profit. From the reinforcement learning perspective, bid and 
ask prices serve as actions, and agents' decisions to buy or sell at those prices provide observations 
that allow the market maker to update its beliefs. As in most reinforcement learning problems, the 
actions (prices) serve the dual role of 1) eliciting information (setting the bid- ask spread too high 
will lead to a lack of trading, yielding little information about the trading public's beliefs) and 2) 
generating reward. 

Das and Magdon-Ismail fSJ present efficient approximate algorithms for performing these up- 
dates for zero profit (ZP) as well as profit maximizing monopolist market makers. In the specific 
algorithm considered, the trader signals are drawn from a Gaussian distribution, and the initial 
market maker belief is also Gaussian. They show that convergence occurs quickly (the market 
maker's uncertainty drops to zero), starting from a state with high information disadvantage. This 
is an advantage over the LMSR market maker which cannot converge in equilibrium unless all trader 
beliefs also converge. When the market maker's initial belief is nearly correct, these algorithms 
deliver zero or near maximum expected profit, another advantage over the LMSR market maker. 
However, in the event of a market shock, the convergence to the new market value is exponentially 
slow - i.e. the market maker is not very adaptive. Figure [T] compares ZP to the LMSR market 
maker in the stylized model that corresponds to the assumptions used to derive the ZP equations, 
illustrating the pros as well as the cons. Clearly what is needed is a market maker with better 
convergence properties than LMSR but better adaptability than ZP. 

3 Market Microstructure 

We consider a prediction market with a single binary outcome stock that trades between and 
100. Presumably, if the event occurs it pays off 100, and if not, it pays off 0. However, this can 
also be thought of as a stock with a liquidating dividend between and 100. At any point in time, 
an arriving trader sees the history of trading in the stock, and the "current price," which can be 
thought of as either the infinitesimal price, the market maker's mean belief about the probability 
of the event occurring, or the middle of the bid-ask spread. The trader then chooses a quantity 
that she wants to buy or sell. The market maker observes the quantity demanded, and sets a price 
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based on this quantity. The trader is informed of this price and can then choose whether or not to 
execute the trade at that price. 

Our market is structured as a pure dealer market, with the market maker as the only price setter. 
Only a single (infinitesimal) spot price is seen by arriving traders. This is a natural formulation for 
the Hanson LMSR market maker, because the actual price is a continuous function of the quantity 
demanded. However, it also serves an important theoretical role for BMM. If an arriving trader 
saw simultaneous bid and ask prices and had a valuation between the two, she would not initiate 
any kind of trade, leaving the market maker with no information about her valuation. Especially 
in times of high uncertainty, this lost information can be a major source of lost liquidity. When an 
arriving trader sees only a single infinitesimal price, she is inclined to "test" the market by placing 
an order on one side or the other. Then, even if she does not execute the trade, the market maker 
can glean valuable information about where her valuation lay (if she places an order but then does 
not execute, the market maker can infer that her valuation lay between the infinitesimal price and 
the quoted price). 



BMM is based on the zero-profit market maker (ZP) mentioned in Section [2] and described in detail 
in [8], with two main innovations: 1) the ability to deal with trade sizes; and, most importantly, 
2) the ability to adapt quickly to market shocks. A trader arrives, observes the spot price pt and 
requests a trade for quantity Q in a direction xt = ±1. xt = +1 means the trader would like to 
buy. For concreteness, we will assume that x% = +1; however, the process is completely symmetric. 
The market maker performs 3 tasks. 

i. Provides a VWAP quote for Q shares; 

ii. Updates its state depending on whether the trade is accepted or canceled; 

iii. Maintains a validity measure for its current beliefs, which is crucial to being able to adapt to 
market shocks. 

We briefly summarize ZP described in [8] first. The market maker's state pt(v) is characterized by 
a Gaussian belief for the value of the market V: Pt(v) = N{m,af). The trader signal is assumed 
to be normally distributed around V, so s ~ N(V, of). The main relevant parameter (see [8]) is 
the information disadvantage of the market maker, pt = crt/a e , the ratio of the uncertainties of 
the market maker and trader. A universal "Q-function" , Q{p) (see |5]) plays an important role in 
quoting prices. Specifically, the spot price is just the market makers mean belief, pt = pt, and the 
ask price is 



This ask gives zero expected profit conditioned on the trade going through; this quoted price does 
not take quantities into account. Described in [8] is a range based update procedure for the market 
maker: if a trader's realized signal is known to lie in the range z~ < s < z + , then the market maker 
updates its Gaussian belief to: 



4 The BMM Algorithm 




Pt+i = pt + ct- 
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where A,B,C are functions of z~ , z + , pt, pt, cr e , the details of which are given in [8]. This range 
based update is used when the trader takes an action (accept or cancel the trade). So, for example, 
if the trader accepts a trade, then s 6 [ask, oo), and so z~ = ask and z + = oo. If the trader cancels 
upon seeing the quoted price, z~ = pt and z + = ask. 



4.1 Quoting a Price for Q Shares 

ZP can only quote a price for a fixed trade size. To be practical, the algorithm needs to quote a 
price for an arbitrary number of shares. The spot price is pt = pt, and assume a trader wants to 
buy Q shares. We implement a heuristic of treating this order as independent orders of a fixed size 
a. There are thus \Q/a~\ independent orders; the sizes ati,ot2, ■ ■ ■ ,ctk are all a, except possibly 
the last one. 

The market maker starts in state p\ = pf, g\ = oj, and imagines the arrival of these k mini- 
orders in sequence; for each mini-order arrival, the market maker quotes the ZP price as in [8]; each 
mini-trade is accepted; the market maker then updates his belief and receives the next mini-trade. 
Specifically, consider mini-trade i, with market maker belief Pi,crf. The price quoted is 



Pi = pi + a t Q{pi) y 1 + pf; 

the trade is accepted, so the market maker updates his belief with z~ = pf, z + = oo: 

B 2 2 f AC + B 2 \ 
p i+ i = pi + ai — , a i+1 = a i \l — 2 I; 

the market maker now processes the next mini-order in the sequence until all the mini-orders are 
processed. Note that these mini-orders are not real, they just describe the process going on in the 
market maker algorithm. Thus, a\,...,ak shares, with ^ a» = Q, get (fictitiously) executed at 
the prices p\ , . . . , p^ . The price quoted to the trader for Q shares is the VWAP for this fictitious 
sequence of executions: 

1 k 

ask = p(Q) = — y~] aiPi. 



Belief Update Since the trader asked to buy, we know that s > pt- The trader is quoted a price 
p(Q), and so based on the traders action, the market maker can update his beliefs to 
using the range update: 

, I accepted z~ = p(Q); z + = oo; 
trade < 

(canceled z = p t ; z + =p(Q). 
We described a buy order, but a sell is entirely symmetric. 



4.2 Adapting to jumps 

The original ZP algorithm leads to constantly decreasing variance of the market maker's belief. 
After a number of trades have been processed the variance and therefore the spreads are significantly 
reduced. While this increases liquidity and encourages further trading towards the true market 
valuation, it is also the root of the market makers inability to adapt to multiple market shocks. 
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In fact, the magnitude of each mean belief update is proportional to the variance of the market- 
maker's belief, large jumps in the true underlying value coupled with a small belief variance, lead 
to very small update values, and the algorithm is exponentially slow in adapting to a jump. 

After a jump, the sequence of trades will be "one-sided", and hence inconsistent with a market 
makers belief of the old valuation coupled with a highly confident low belief variance. The simple 
solution to this is to allow the market maker to become less confident as he see a sequence of 
extremely one sided trades, i.e. an inconsistent sequence of trades. To accomplish this, we define 
a consistency index C(history), which measures exactly how likely the recent history of trades 
observed under the current uncertainty level is, as compared to a higher uncertainty. An intuitive 
solution is to increase the market maker's belief variance during periods of inconsistency. 

Specifically, BMM keeps track of a fixed window of previous trades (including canceled trades), 
along with the z + and z~ values that are inferred from those trades. Then, at a particular time 
step, the probability of a sequence of trades over a window of size W, can be computed as: 



The intuitive solution is to compare this probability against a fixed threshold; if the probability 
is too small, we are in an inconsistent regime, and so we increase the market maker's uncertainty 
level (increase the variance). However, this solution is problematic because the threshold is highly 
sensitive to the choice of window size and particular features of the trade sequence. Instead, we 
make a relative comparison with the same probability computed at twice the uncertainty. We thus 
define our consistency index 



If C > 0, we increase at, specifically at+i = 2a t . The choice to double the variance is arbitrary, 
and any multiplier greater than 1 would do. Though we have only tried the multiplier 2, we expect 
that since this is a relative measure of consistency, the results would be robust to the choice of 
multiplier, unlike with the use of a fixed threshold. 

This algorithm takes advantage of the fact that more "even" sequences of trades are more likely 
when the variance is lower, while sequences that are heavily biased in one direction or the other 
become more likely with higher variance. The key parameter for this algorithm is the window size 
W, which controls the balance between how stable the market maker is at equilibrium and how fast 
it can adapt to changes. The window size W also now becomes the dominant factor in measures 
like average spread, so that the particular value of a e becomes unimportant. 

5 Simulation Experiments 

In order to test the market making algorithms and elicit their general properties, we conducted 
extensive simulation experiments before deploying them in situations with live human trading. The 
goals were to (1) ensure the adaptive capabilities of BMM (2) compare BMM and Hanson's LMSR 
MM on the basis of profit/loss, average spreads, and price discovery, and (3) calibrate parameters so 
that the live trading tests could be done with market makers that were similar in average spreads. 

The simulation environment is structured as follows. Each trading simulation consists of 200 
discrete time steps. There is an underlying "true value" process. The initial true value is drawn 




))dv 



C (history) = L(fj, t , 2a t ) - L(/z t , cr t ); 
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Initial convergence of MM's be- 
Spot price versus true market liefs (spread) shown by the Spread convergence for window 
value. width of the gray region (log size 10. 

scale) . 

Figure 2: Behaviors of the market makers in an example simulation. The first two figures show 
BMM with window size 5, and the third shows window size 10. BMM is clearly capable of adapting 
rapidly to changing behavior in the trading population, but at the same time shows less jumpy and 
unstable behavior at equilibrium than the LMSR MM. This behavior is explained by the spread 
(which is a function only of the variance of the MMs belief). The spread starts off high, and 
increases around times of uncertainty, allowing the mean to move more quickly. At the same time, 
this can create occasional instabilities even when the underlying population mean has not jumped 
(note some of the periods of increasing spread when the true value is stable). 



from a Gaussian distribution with mean 50 and standard deviation 12 (in general, all values are 
truncated at and 100 whenever that may be an issue). Then, at every time step, there is a 
probability pj that the true value jumps. We consider two different types of jumps. In the first 
type, which is more realistic, the amount of the jump is drawn from a Gaussian distribution with 
mean and variance er?. In the second type, which is meant to simulate a very problematic case 
for an information based market maker, the new value is itself drawn uniformly at random between 
and 100. At any point in time, an arriving trader receives a valuation wt drawn from a Gaussian 
distribution with mean equal to the true value at that time, and variance a^. If wt exceeds the 
current infinitesimal price, the trader initiates a buy order, and if it is less the trader initiates a 
sell order. The quantity is drawn at random from an exponential distribution with rate parameter 
a q ^ In our experiments, we set pj = 0.01, Oj = 5, and a e = 5. a q is set to 0.05 so that the mean 
trade size is 20. The b parameter for the LMSR market maker was set to 125 and the window size 
parameter for BMM was set to 5. These choices of the MM parameters were in order to make the 
average spread approximately equal in the Gaussian jumps case, and were then used again for the 
initial live trading experiments described in the next section. 

Figure [2] gives some intuition into the behavior of BMM as compared with LMSR. This is for a 
single experiment, and shows that BMM can adapt rapidly to changing valuations in the trading 
population, while at the same time settling into periods of low spreads and stable behavior at 

3 This random quantity model is frequently used in models of zero-intelligence trading and models from the 
econophysics literature (e.g. [9]). 



11 





Gaussian Shocks 


Uniform Shocks 




BMM 


LMSR 


BMM 


LMSR 


Profit 


2081.35 


-2457.30 


603.40 


-1897.98 


Max Loss 


9479.82 


8662.32 


50183.77 


8384.42 


Spread 


1.42 


1.35 


1.79 


1.40 


RMSD 


2.92 


5.38 


8.78 


10.79 



Table 1: Performance of BMM and the LMSR MM in simulated trading 

equilibrium. The typical behavior is to start off with a high variance (and hence high spread), and 
then quickly converge to a low variance regime. When a jump in the population belief occurs, the 
market maker can quickly pick up on that fact using the algorithm described previously, because 
the sequence of trades it sees is usually heavily biased in one direction, which would be more likely 
to occur if the market maker's beliefs had a higher variance (in contrast, series of trades that are 
more balanced are more likely to occur in a model with lower variance, since the probability mass 
is more concentrated in the "likely" region). Because of the adaptivity, in a long stable period 
there will be times when the variance (and spread) will increase even though no true change has 
occurred. This becomes more likely as the variance gets lower. 

It is important to point out that this behavior is general. For about the same average spread, 
BMM can in general achieve better market properties in terms of stability at equilibrium as well 
as profit. In this particular simulation, the average quoted (half) spread for BMM was 1.23 and 
its profit was 3080.29. The average quoted (half) spread for the LMSR MM was 1.91 and its 
profit was —956.27. Table [T] demonstrates this fact more generally by showing results from 1000 
simulations. In addition to the average profit and spread, this table also reports the root mean 
square deviation of the infinitesimal price from the true value (population mean) at any given point 
in time (a measure of price discovery), and the single worst loss suffered by the market makers in 
1000 simulations (in both cases the single worst loss suffered by the LMSR market maker is close 
to the theoretical bound of 8664.34. BMM performs better on average. However, it is worth noting 
that, as the probability of a jump goes up, especially in the case where new valuations are drawn 
uniformly at random, the loss suffered by BMM increases, so it may not be the best choice for 
highly unstable environments. 

6 Live Trading 

We now present an experimental design for comparing market makers in a live trading setting 
with human subjects. Human subject experiments by their very nature use small samples; further, 
human subjects are diverse and very rapid learners, whose attention cannot reliably be maintained 
for extended time periods. This poses several challenges to live trading experiments when trying 
to compare market makers. 

(i) Two comparably sized groups can display vastly different behaviors due to inherent diversity 
in backgrounds, skill sets and tendencies among human subjects. 

(ii) Human subjects, being natural learners, build biases very quickly. So, for example, if you run 
an experiment for the first time with a market value of (say) 0.7, traders may take some time 
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to become accustomed to the trading task. If you run exactly the same experiment again, 
it is possible that the second time around, the traders will display more intelligent behavior, 
with perhaps even a bias that the value is around 0.7, having "generalized" from the previous 
experiment. 

The implications are that to get useful results, the live trading experiment should use the same 
group of traders simultaneously to compare a pair of market makers. Further, the market makers 
should be compared in a completely symmetric way, using an intuitive interface. 

We use a very simple trader interface, similar to a web-trading interface of a typical online 
broker (see Figure [3]). Traders are allowed to only place market orders, and in order to elicit 
information, only the spot price is displayed. A trader can then offer a trade (buy or sell) and a 
desired quantity, upon which the trader is quoted a (volume weighted average) price. The trader 
may either accept or cancel the trade. 

6.1 Experimental Design 

There are two markets, lr and tb, which are based on the 2 dimensional random walk illustrated 
in Figure |4| The 2 dimensional random walk is two independent 1 dimensional random walks: 



horizontal (lr) and vertical (tb). Each random walk is a classic Gambler's Ruin problem |10j . The 
starting position (indicated by the dotted red circle) is (xo,yo)i an d there are two probabilities, 
Plrj the probability of moving right in the horizontal dimension, and p TB , the probability of moving 
down in the vertical dimension. The random walk (x,y) is bounded in the grid [— S, S] 2 . So if 
\x\ > S, the x-coordinate of the random walk is restarted at xq (the y-coordinate is left unchanged) 
and similarly if \y\ > S, the y-coordinate of the random walk is restarted at yo (the x-coordinate is 
left unchanged). 

The values of the markets lr and TB are defined before any particular experiment, based on 
how often the ball hits the right edge before the left edge, or the bottom edge before the top edge. 
The probability that the ball hits the right (resp. bottom) edge before the left (resp. top) edge can 




Figure 4: 2- dim random walk 
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be computed analytically [TO]. In terms of p LR , p TB , xq, yo, S these values are (For p ^ 



x S-x _ \2S \ s ~vo _ \2S 

f " LR t 7 TB ^TB 



T T ' 'LK ' "LR T T 

' T,R — I TTTT^ i Vn 



where A = — p). Traders are allowed to simultaneously trade in both markets lr and TB. 
For the experiments, we set p LR = p TB = p and xq = yo = z. Thus, other than one market being 
visually represented vertically and one horizontally, the two markets are completely symmetric. 



Trader Signals The traders see a realization of the random walk unfolding over time. As shown 
in Figure [4j the number of times the walk has hit the left, right, top and bottom edges is shown, 
together with how much time is left. A trader can estimate V^r and V TB from these numbers; 
for example, from the figure, we can make out from this partial realization of the random walk 
that V LR « || = 0.75 and V TB ~ 75 = 0.60. Although these are realizations of the same random 
process, we immediately see that the trader is getting a noisy signal of the variable on the basis 
of which the market pays off (as t — > 00, traders would have perfect information that determines 
the payoff). This signal improves with time as more information is revealed; in particular, in our 
example, the error in the traders signal decreases in proportion to \j\ft- This process of gradual 
information revelation is similar to what goes on in real prediction markets, with traders getting 
better information over time. 



Market Shocks In a normal equilibrium setting the parameters p, S, z are fixed. We can institute 
a market shock during the random walk by changing one or more of these parameters. Changing 
these parameters can reflect different types of market shocks in the real world - for example, if p 
changes, there is no visible cue, and traders have to infer a change in the underlying dynamics from 
observables. 



6.2 Description of Experiments 

Our data were collected in three distinct trading sessions; we used results from the first two sessions 
in order to improve the design of the second session. All the traders in our experiments were 
relatively sophisticated; they all had prior experience with the trading interface and knowledge 
of prediction markets. In each case they sat in the same room and traded using the web-based 
interface on their personal laptops. 

The first two sessions were individual experiments in which students from a graduate-level 
Computational Finance class were recruited to participate. 11 and 9 students respectively chose to 
participate in the two experiments. Participants were incentivized with gift certificates: the trader 
with largest return received $15; the second best received $10 and the three next best traders $5 
each. 

The third session was an educational deployment of the market as part of a graduate / advanced 
undergraduate class on E-Commerce. Students were studying prediction markets and participated 
in four trading games during the class. They were incentivized with the opportunity to earn extra 
credit in the class. 17 students participated. 10 points of total extra credit were allocated for the 
experiment, with the 10 points divided proportionally among all traders who overall made profit 
in the experiments. 

4 For p = I one has to take a limit (eg. Vlb = S ^~g ) 
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In each case, the LMSR based market maker was configured with the loss parameter b set to 
125. The information-based market maker was configured to begin with belief fit = 50, at = 12, 
and estimation of the trader noise given by a t = 5. The window of trade history for the adaptive 
mechanism was set to 5 for the first two experiments and to 10 for the last four. In the first two 
experiments traders started with 100, 000 units of currency and shares, and were allowed to take 
both long and short positions in each market. In the four subsequent experiments, traders began 
with an initial endowment of 10, 000 units of currency, and 200 shares, but were not permitted to 
take short positions. We flipped the market makers being used in the TB and LR markets between 
experiments. 

We now provide details of the experiments. For experiments 1 and 2, traders were told that 
there may or may not be a change in the underlying parameters governing the random walk. The 
conditions for the remaining experiments are described below. In all cases except for Experiment 
2, final payoffs were based on the analytically computed probabilities described above. A summary 
of the parameters used in each experiment are shown in Table [2] below. 

Experiment 1: Equilibrium Each trader viewed an independent realization of the random 
walk for 10 minutes, and so the traders had their own personal information set based on the 
random walk they were seeing, as well as the price dynamics which carried information regarding 
the realizations that other traders were seeing. 11 traders participated. 

Experiment 2: Common Information Shock In this experiment, all traders viewed the 
same random walk realization, projected on a screen. They were told that the payoff of the markets 
would be the actual realized ratios of the two random walks, rather than the analytically computed 
probabilities. The parameters governing the random walk were "shocked" at the 5 minute mark. 
In this case, the traders' information gradually becomes completely correct, and the market maker 
is eventually trading against perfectly informed traders. 

Experiment 3: Limited Information Equilibrium This experiment was similar to Exper- 
iment 1, except for the fact that traders only saw their personal realization of the random walk for 
2 minutes. They were allowed to trade for 10 minutes. 17 students participated. 

Experiments 4, 5, and 6: Equilibrium With Probabilistic Shocks Before these exper- 
iments, students were told that they would be participating in 3 consecutive games. In each of 
these games, the random walk would start off with some combination of parameters. With a 50% 
chance, these parameters would change between minutes 3 and 7 of the random walk. Traders were 
not told whether or not there would be a jump in a particular experiment. A coin was flipped for 
each of the three experiments. There was no shock (change in parameters) in Experiments 4 and 
5, while there was a shock in Experiment 6 (therefore we call it IndivInfoShock below). Trading 
went on for 10 minutes. 17 students participated. 

6.3 Results 

Figure [5] shows the main results of the experiments (the analogs of the simulation data in Figure [TJ, 
and Table[3]shows some statistics on the price processes. There are various interesting phenomena in 
the individual experiments, discussed below, but the big picture is relatively clear. BMM dominates 
LMSR in terms of profit made in five of the six live experiments, while at the same time producing 
a more stable price process, with better price discovery, as measured by distance from the "true" 
value (RMSD values in Table [3j The behavior of BMM is improved in experiments 3 through 6, 
which were run with a longer adaptive window of 10, leading to more stability (and potentially 
slower adaptivity). While higher values of the b parameter for LMSR would lead to improved 
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(c) Limitedlnformation 



(d) Equilibrium(4) 



■ ZPIMM 
LMSR 

True Value 
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- ZPIMM 
■ LMSR 
True Value 
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(e) Equilibrium(5) 



(f) IndivInfoShock 



Figure 5: Summary behavior of LMSR and BMM in live trading experiments. Experiments (a) and 
(b) are from one deployment in which BMM used a window size of 5 and (c) through (f) from a 
later one in which BMM used a window size of 10. In general, BMM exhibits more stable behavior 
than LMSR. Additionally, In the "shock" experiments (b) and (f), BMM is now able to adapt as 
well as, if not better than, LMSR. 
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P 


S 


z 


V 


Equilibrium 


0.600 


4 


-1 


0.7322 


CommonlnfoShock 


0.600 


2 


-1 


0.5846 


jump to 


0.600 


2 


+ 1 


0.1231 


Limitedlnformation 


0.764 


4 


-3 


0.6912 


Equilibrium(4) 


0.533 


4 


-3 


0.1897 


Equilibrium(5) 


0.866 


4 


-3 


0.8453 


IndivInfoShock 


0.826 


4 


-3 


0.7890 


jump to 


0.516 


4 


-2 


0.2999 



Table 2: Random walk parameters for each of the 6 human subject experiments. 





Profit 


Spread 


RMSD 


RMSDeq 




LMSR 


BMM 


LMSR 


BMM 


LMSR 


BMM 


LMSR 


BMM 


Equilibrium(l) 


-1350.12 


47231.77 


3.12 


4.04 


4.92 


7.66 


3.73 


3.01 


CommonlnfoShock 


-1510.89 


8972.50 


3.06 


3.21 


20.67 


15.98 


16.51 


6.76 


Limitedlnformation 


-1602.14 


4083.95 


1.61 


0.49 


14.43 


2.15 


14.56 


0.93 


Equilibrium(4) 


-2619.07 


-10588.86 


1.81 


0.95 


21.67 


23.13 


14.82 


17.05 


Equilibrium(5) 


-3168.55 


9134.58 


1.42 


0.51 


11.18 


1.5 


8.15 


1 


IndivInfoShock 


-92.29 


20226.44 


1.89 


0.89 


8.87 


6.47 


6.88 


6.6 



Table 3: Summary statistics of MM performance in the live trading experiments. Spreads are 
the offered spread for 40 shares (the average trade size observed) at each point in time when a 
trade executed. RMSD is the root mean square deviation of the MM's belief from the true value. 
RMSD eq is the same metric evaluated after "convergence," defined for convenience as the time 
period halfway between the last change to the true value and the end of the trading period. 

stability (and slower adaptivity), this would come at the cost of making even greater losses. 

Experiments 1 and 2, and lessons learned In the Equilibrium(l) experiment, there are some 
severe fluctuations at about the 75 sec and 200 sec marks. The fluctuations around the 75 sec mark 
are probably due to individuals who had outlier realizations early on. The fluctuations around 
the 200 sec mark are due to a single irrational "rogue" trader who was willing to buy at a price 
of 100. Unfortunately, since there is no penalty for random wild trading (unlike in real financial 
markets), such behavior is bound to arise with human experiments. Discounting these anomalous 
trades, BMM converges quite nicely to equilibrium, as does LMSR (except for its characteristic 
oscillations). Further, in the MarketShock experiment, BMM now adapts as fast if not faster than 
Hanson. 

The BMM profit in the Equilibrium(l) experiment is a little misleading because about 30,000 
of it was due to the rogue trader; BMM does what it is supposed to do though, by adapting 
and making profit based on its Bayesian learned valuation. This wild trader also accounts for the 
increased RMSD of BMM in this experiment. After the market equilibrates and finds the true 
value, the RMSD of BMM dominates LMSR (the RMSD cq row in Table [3f. Similarly, when the 
market is close to equilibrium in the MarketShock experiment (in this case, after seven and a half 
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(a) Equilibrium(l) 



(b) CommonlnfoShock 



Figure 6: Dynamic behavior of LMSR and BMM spread in Experiments 1 and 2. BMM often 
increases its spread in response to unlikely trade sequences, making it highly adaptive, while sac- 
rificing some stability. 



minutes of trading time in total; since the jump occurs after five minutes, we give the market half 
of the remaining time to equilibrate) the RMSD of BMM dominates LMSR by a significant margin. 

These experiments reveal a couple of interesting facts. First, the behavior of some rogue traders 
can seriously impact outcomes. In this case, it seems that, when given large initial endowments and 
the ability to sell short, some traders use their market power to full effect without worrying about 
profit. So we decided to give people more "reasonable" endowments in the future, including an 
endowment of stock to start with, and prohibit short-selling. This likely leads to a psychologically 
more understandable scenario for participants, and less possibility for arbitrary manipulation by 
traders who are psychologically uninvested in the outcome. 

Second, the spreads and behavior of BMM were somewhat less stable than we had expected 
based on simulation. Figure [6] shows that BMM often increases the spread in response to market 
conditions, even though there are relatively few shocks in the system. While this still yields good 
behavior, we hypothesized that tweaking the window parameter would lead to more stable behavior 
without sacrificing adaptivity too much. Therefore, we changed the window size to 10 for the next 
set of experiments. 



Experiments 3 through 6 Experiments 3 through 6 demonstrate the typical behaviors of BMM 
and LMSR clearly. There are a couple of interesting details that emerge from the experiments. First, 
in Experiment 4 (Equilibrium(4)), convergence to the true value is very slow for both LMSR and 
BMM. While LMSR comes close to the true value in the last few seconds before the end of trading, 
BMM fails to do so. We hypothesize that this is because this market was the only one in which 
the true value was below the starting value of 50, and thus necessitated people selling their initial 
endowment to get to the true value. In this case, BMM also takes a fairly substantial loss, because 
it was misled by the trading behavior. 

Second, in the IndivInfoShock experiment (number 6), while traders were told that the true 
value would only be the true value after the shock, later interviews with participants revealed that 
they thought the true value would be the average of the two true values. Therefore, the stock ended 
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(c) Equilibrium(5) (d) IndivInfoShock 

Figure 7: Dynamic behavior of LMSR and BMM spread in Experiments 3 through 6. BMM is now 
significantly more stable, but (d) shows it is still quick to adapt. 

up trading at around 60, instead of the final true value of 30. 

In both these cases, it is nice that the symmetry of the experimental design enables fair com- 
parison between LMSR and ZPIMM: trader behavior leads to anomalies for both market makers. 
Figure [7] shows the behavior of the spreads for the two market makers, demonstrating that BMM 
is significantly more stable in these experiments, while still reasonably adaptive, as evidenced by 
Experiment 6 (IndivInfoShock). 

Number of trades The table below provides a summary of the comparison of the LMSR and 
BMM market makers in terms of the number of confirmed buy or sell trades executed in each 
experiment. The numbers are roughly comparable, although LMSR usually comes in a little bit 
higher, presumably because it provides profit opportunities due to price fluctuations when BMM 
has stabilized. 
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# Buy/Sell 
LMSR BMM 



# Traders 



Equilibrium 

CommonlnfoShock 

Limitedlnformation 



119 146 

213 131 

116 63 

109 113 

118 85 

113 121 



11 

9 

17 
17 
17 
17 



Equilibrium(2) 
Equilibrium(3) 
IndivInfoShock 



Summary Our live trading experiments demonstrate several key facts. First, while LMSR has 
nice theoretical properties that suggest it will converge to rational expectations equilibria, in prac- 
tice this is asking an awful lot of the participants in the market. As long as traders' posterior beliefs 
do not converge to a single point, there will remain trading incentives, and this is in evidence in 
all our experiments. LMSR suffers from characteristic fluctuations in the spot price even after it 
should have attained equilibrium. BMM, on the other hand, provides a tighter belief once it has 
converged, and has attractive potential to make markets without losing money, or even at a profit. 
It manages this while providing superior price discovery and spread properties in our live trading 
experiments. 

Experiment 4 (Equilibrium(4)) provides evidence that BMM may sometimes suffer high losses, 
especially when the market behaves strangely. While occassional such instances are not a huge 
problem, it will be important to monitor and understand the circumstances that can lead to high 
losses so that we can ensure that they cannot be reproduced by manipulators intentionally deceiving 
the market maker. 

7 Conclusion 

We have presented an adaptive, information based Bayesian market maker BMM. In simulation as 
well as in live trading, when controlling for liquidity (as measured by spread), BMM demonstrates 
significantly better convergent behavior at equilibrium than Hanson's LMSR market maker, while 
being equally adaptive to changes in the market's valuation of the security. BMM also provides a 
meaningful quantitative posterior probability distribution for the value of the security being traded. 
Further, it does not lose money (in fact it typically makes money) in both the idealized simulated 
markets and live trading, implying that it could provide substantially better liquidity at lower cost 
than Hanson's LMSR. The caveat is that, unlike Hanson's LMSR market maker, BMM is not loss 
bounded. BMM thus provides a real alternative to Hanson's LMSR for market making in real 
information markets, with many potential benefits. 

Our second goal was to present a symmetric, fair experimental design for comparing two market 
makers in a live trading setting. We have only begun to explore the possibilities of this paradigm, 
which offers a realistic trading environment in which traders gradually get information as they 
trade against the dealer. The design allows one to study market shocks with and without visible 
cues, to study convergence and adaptation of market makers, and to study real trader behavior - 
how do traders really trade given their valuation and the market price? 

There has been recent recognition in the literature of some of the problems with LMSR; in 
particular, Othman et al have noted its liquidity-insensitivity and the difficulty of setting the 
b parameter appropriately [16]. They have proposed an alternative approach which varies the 
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b parameter - this is another potential alternative to LMSR that is important to explore and 
characterize, and our experimental platform provides a good method for testing this market maker 
as well. 

It is worth noting that all the experiments described in this paper are information aggregation 
experiments. Many prediction markets are important because of their information dissemination 
role (see for example, Othman and Sandholm's recent work on a market to predict the opening of 
the new Computer Science building at Carnegie-Mellon University |15j). In these cases, only one 
or a few insiders have knowledge of the true value, and the market's goal is to incentivize these 
insiders to reveal their information. Testing market makers in such settings is important. 

Future work on developing BMM further should focus on characterizing situations where it can 
potentially make significant losses, and attempt to mitigate losses in these situations. In particular, 
BMM's susceptibility to manipulation from a trader who understands and attempts to beat the 
algorithm is not yet well understood. Further experimental evaluation is also necessary. Another 
interesting open question is whether we can formulate precisely the tradeoff between convergence, 
loss and adaptability for market makers. 
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